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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 

In re Application of: 

TIMOTHY MERRICK LONG 

Application No. : 09/612,440 

Filed: July 7, 2000 

For: COMBINING A PLURALITY 
OF IMAGES AND 
TRANSFORMING AN IMAGE 
HAVING ASSOCIATED 
META-DATA 

Commissioner for Patents 
Washington, D.C. 20231 

CLAIM TO PRIORITY 

Sir: 

Applicant hereby claims priority under the 
International Convention and all rights to which he is 
entitled under 35 U.S.C. § 119 based upon the following 
Japanese Priority Application: 

PQ 1525 filed on July 9, 1999. 

A certified copy of the priority document is 

enclosed. 
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Examiner: N.Y.A. 
Group Art Unit: 2771 
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Applicant's undersigned attorney may be reached in 
our New York office by telephone at (212) 218-2100. All 
correspondence should continue to be directed to our address 
given below. 

Respectfully submitted, 

Attorney for Applicant 
Registration No. "T^? 

FITZPATRICK, CELLA, HARPER & SCINTO 
30 Rockefeller Plaza 
New York, New York 10112-3801 
Facsimile: (212) 218-2200 
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Patent Office 
Canberra 



I, ANNA MAIJA EVERETT, ACTING TEAM LEADER EXAMINATION 
SUPPORT & SALES hereby certify that annexed is a true copy of the 
Provisional specification in connection with Application No. PQ 1525 for a 
patent by CANON KABUSHIKI KAISHA filed on 09 July 1999. 

I further certify that pursuant to the provisions of Section 38(1) of the Patents 
Act 1990 a complete specification was filed on 06 July 2000 and it is an 
associated application to Provisional Application No. PQ 1525 and has been 
allocated No. 45063/00. 




WITNESS my hand this 
Thirteenth day of July 2000 



ANNA MAIJA EVERETT 
ACTING TEAM LEADER 
EXAMINATION SUPPORT & SALES 



CERTIFIED COPY OF 
PRIORITY DOCUME^ 
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AUSTRALIA 
Patents Act 1990 

PROVISIONAL SPECIFICATION FOR THE INVENTION ENTITLED : 

Digital Image Meta-Data 



Name and Address 

of Applicant: Canon Kabush1k1 Kalsha, Incorporated In Japan, of 30-2 f 

Shlmomaruko 3-chome, Ohta-ku, Tokyo, 146, JAPAN 



Name of Inventor: Timothy Merrick Long 

This Invention 1s best described In the following statement: 



DIGITAL IMAGE META-DATA 



Field of Invention 

The current invention relates to digital image meta-data and the automatic 
merging of meta-data fields in particular. 

Description of Background art 

Many digital image formats include meta-data which records aspects of an image 
that are in addition to the raw digital image pixel data. Examples of such attributes 
include a copyright notice, type of capture device, flash usage, type of lens, date of 
capture, programs the image has been edited with, regions of interest, and more. In many 
image formats there are both well known meta-data items which, because of their 
standard-mandated identifiers and format, can be programmatically processed, as well as 
application specific meta-data items. These application specific meta-data items may be 
related to only one or a few programs or may be end-user invented and thus not 
programmatically interpretable by any particular application. 

There exist many computer application programs which combine digital images 
in various ways, or transform a single image. Examples include image editors such as 
Adobe PhotoShop. Other examples include panorama stitching applications that combine 
two or more partially overlapping images of the same scene into a single larger digital 
image. When these applications produce a new digital image from one or more original 
digital images they must make decisions on what meta-data from the original image(s) is 
passed to the new image, what meta-data is discarded, or modified in some way. In the 
current state of the art, meta-data specific copying actions may be taken on meta-data 
elements which have well known characteristics. For example, a program may know to 
preserve all copyright notices from all source images in the composite image because the 
characteristics of copyright notices are known to the application designer. But for 
application specific meta-data elements that are not known to the merging application 
designer there is currently no method of coding the application to have knowledge of how 
these meta-data elements should be treated. This invention solves this problem. 

SUMMARY OF THE INVENTION 

According to one aspect of the invention there is provided a method of 
combining image meta-data comprising the steps of: providing a plurality of images, 
each image having associated therewith at least one meta-data element; storing at least 
one self-describing attribute tag with said meta-data element for each said image, wherein 
said attribute tag describes an action to be taken with said meta-data element of the first 
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one of said images and a corresponding meta-data element of a second one said images 
are combined; and combining the meta-data element of the first one of said images and a 
corresponding meta-data element of a second one of said images in accordance with said 
action. 

According to another aspect of the invention there is provided apparatus for 
implementing the abovementioned method. 

According to another aspect of the invention there is provided a computer 
program product comprising a computer program for implementing the abovementioned 
method. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 shows a general purpose computer for implementing the preferred method. 

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION 

In the current invention any digital image meta-data elements may be augmented 
with an attribute which describes how that meta-data element from multiple images 
should be merged when the images are merged, or a new transformed image is created. In 
the preferred embodiment the meta-data associated with a digital image is recorded in 
XML format in a portion of the digital image. But is should be noted that many other 
methods of storing meta-data in a digital image exist and could be used. For example, the 
tag method of the TIFF image file format and the Structured Storage mechanism of the 
Flashpix image file format. And example of meta-data in XML format could be: 
<?xml version= M 1.0"> 
<PHOTO> 

<Photo_Number>62</Photo_Number> 
<Lens>EF85mm f/1.2L USM</Lens> 
<Creator>Takashi Mochizuki</Creator> 
<Copyright>Takashi Mochizuki, 1999</Copyright> 
<ModelRelease desc="Man in blue suit" 

release="yes" ref="MR45621" /> 
<EyeLocation>123 546 130 600</EyeLocation> 

< /PHOTO > 

The present invention adds attributes to each XML element of the digital image 
meta-data to describe how that element should be combined with like elements of other 
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digital images when those images are combined. Similar mechanisms could be employed 
in on-XML based meta-data storage formats. The preferred embodiment allows the 
following merging attributes with the given interpretation: 
discard 

These elements should be discarded and no element of this name placed in the final 
image upon transformation or combination. An example of an element of this type is a 
region of interest which will become invalid through most transformations, 
keepboth 

These elements should each be kept. Both should be placed in the final image. 
An example of this type of element is a legal notice such as a model release note, 
keepsame 

These elements should be kept as a single attribute if they have the same value, 
else both should be discarded. Example of these elements are camera capture 
characteristics such as whether a flash was used and date of capture, 
keepone 

One of these elements should be chosen and copied to the final image at the 
application's discretion. Examples of these elements are captions or titles. 
Clearly other merging attributes are possible. 

In the preferred embodiment the default action when no explicit merge attribute 
is supplied with a meta-data element is the keepsame action. 

An example of some digital-image meta-data with these attributes incorporated 
could be: 

<?xml version="1.0"> 
<PHOTO> 

<Photo__Number>62</Photo_Number> 
<Lens>EF85mm f/1.2L USM</Lens> 

<Creator merge="keepboth">Takashi Mochizuki</Creator> 
<Copyright merge="keepboth ,, >Takashi Mochizuki, 1999</Copyright> 
<ModelRelease merge="keepboth" desc="Man in blue suit" 
release="yes" ref="MR45621" /> 
<EyeLocation merge="discard"> 1 23 546 130 600</EyeLocation> 
</PHOTO> 

The preferred method is preferably practiced using a conventional general-purpose 
computer system 100, wherein the processes may be implemented as software, such as an 
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application program executing within the computer system 100. In particular, the steps of 
the preferred method are effected by instructions in the software that are carried out by 
the computer. The software may be divided into two separate parts; one part for carrying 
out the preferred methods; and another part to manage the user interface between the 
5 latter and the user. The software may be stored in a computer readable medium, 
including the storage devices described below, for example. The software is loaded into 
the computer from the computer readable medium, and then executed by the computer. A 
computer readable medium having such software or computer program recorded on it is a 
computer program product. The use of the computer program product in the computer 

lo preferably effects an advantageous apparatus for implementing the preferred method in 
accordance with the embodiments of the invention. 

The computer system 100 comprises a computer module 101, input devices such 
as a keyboard 102 and mouse 103, output devices including a printer 115 and a display 
device 114. A Modulator-Demodulator (Modem) transceiver device 116 is used by the 

is computer module 101 for communicating to and from a communications network 120, for 
example connectable via a telephone line 121 or other functional medium. The 
modem 1 16 can be used to obtain access to the Internet, and other network systems, such 
as a Local Area Network (LAN) or a Wide Area Network (WAN). 

The computer module 101 typically includes at least one processor unit 105, a 

20 memory unit 106, for example formed from semiconductor random access memory 
(RAM) and read only memory (ROM), input/output (I/O) interfaces including a video 
interface 107, and an I/O interface 113 for the keyboard 102 and mouse 103 and 
optionally a joystick (not illustrated), and an interface 108 for the modem 116. A storage 
device 109 is provided and typically includes a hard disk drive 110 and a floppy disk 

25 drive 111. A magnetic tape drive (not illustrated) may also be used. A CD-ROM 
drive 112 is typically provided as a non-volatile source of data. The components 105 
to 1 13 of the computer module 101, typically communicate via an interconnected bus 104 
and in a manner which results in a conventional mode of operation of the computer 
system 100 known to those in the relevant art. Examples of computers on which the 

30 embodiments can be practised include IBM-PC's and compatibles, Sun Sparcstations or 
alike computer systems evolved therefrom. 

Typically, the application program of the preferred embodiment is resident on 
the hard disk drive 110 and read and controlled in its execution by the processor 105. 
Intermediate storage of the program and any data fetched from the network 120 may be 

CFPP1402 Standard06 470309 l:\ELEC\ClSRA\STANDARD\STANDARD06\470309.doc 



5 

accomplished using the semiconductor memory 106, possibly in concert with the hard 
disk drive 110. In some instances, the application program may be supplied to the user 
encoded on a CD-ROM or floppy disk and read via the corresponding drive 112 or 111, 
or alternatively may be read by the user from the network 120 via the modem device 1 16. 
Still further, the software can also be loaded into the computer system 100 from other 
computer readable medium including magnetic tape, a ROM or integrated circuit, a 
magneto-optical disk, a radio or infra-red transmission channel between the computer 
module 101 and another device, a computer readable card such as a PCMCIA card, and 
the Internet and Intranets including email transmissions and information recorded on 
websites and the like. The foregoing is merely exemplary of relevant computer readable 
mediums. Other computer readable mediums may be practiced without departing from 
the scope and spirit of the invention. 

The preferred method / may alternatively be implemented in dedicated hardware 
such as one or more integrated circuits performing the functions or sub functions of the 
method. Such dedicated hardware may include graphic processors, digital signal 
processors, or one or more microprocessors and associated memories. 

The foregoing describes only one embodiment/some embodiments of the present 
invention, and modifications and/or changes can be made thereto without departing from 
the scope and spirit of the invention, the embodiment(s) being illustrative and not 
restrictive. 

In the context of this specification, the word "comprising" means "including 
principally but not necessarily solely" or "having" or "including" and not "consisting only 
of. Variations of the word comprising, such as "comprise" and "comprises" have 
corresponding meanings. 
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The claims defining the invention are as follows: 



1 . A method of combining image meta-data comprising the steps of: 

providing a plurality of images, each image having associated therewith at least 
one meta-data element; 

storing at least one self-describing attribute tag with said meta-data element for 
each said image, wherein said attribute tag describes an action to be taken with said meta- 
data element of the first one of said images and a corresponding meta-data element of a 
second one said images are combined; and 

combining the meta-data element of the first one of said images and a 
corresponding meta-data element of a second one of said images in accordance with said 
action. 



Dated 9 July, 1999 
Canon Kabushiki Kaisha 

Patent Attorneys for the Applicant/Nominated Person 
SPRUSON & FERGUSON 
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FIG- 1 



